Neural network based physical condition evaluation of electronic devices, and associated systems and methods

ABSTRACT

Systems and methods for evaluating the physical and/or cosmetic condition of electronic devices using machine learning techniques are disclosed. In one example aspect, an example system includes a kiosk that comprises an inspection plate configured to hold an electronic device, one or more light sources arranged above the inspection plate configured to direct one or more light beams towards the electronic device, and one or more cameras configured to capture at least one image of a first side of the electronic device. The system also includes one or more processors in communication with the one or more cameras configured to extract a set of features of the electronic device and determine, via a first neural network, a condition of the electronic device based on the set of features.

CROSS-REFERENCE TO RELATED APPLICATIONS

This patent application claims priority to and the benefit of U.S.Provisional Patent Application No. 62/807,165, entitled “NEURAL NETWORKBASED PHYSICAL CONDITION EVALUATION OF ELECTRONIC DEVICES, ANDASSOCIATED SYSTEMS AND METHODS,” filed Feb. 18, 2019. The entirecontents of the above-mentioned patent application are incorporatedherein by reference in their entirety.

TECHNICAL FIELD

The present technology is generally directed to evaluating the conditionof mobile phones and/or other electronic devices, such as evaluating thepresence, quantity, and/or distribution of surface scratches or cracksin such devices, based on machine learning techniques.

BACKGROUND

Consumer electronic devices, such as mobile phones, laptop computers,notebooks, tablets, MP3 players, etc., are ubiquitous. Currently, thereare over 6 billion mobile devices in use in the world; and the number ofthese devices is growing rapidly, with more than 1.8 billion mobilephones being sold in 2013 alone. There are now more mobile devices inuse than there are people on the planet. Part of the reason for therapid growth in the number of mobile phones and other electronic devicesis the rapid pace at which these devices evolve, and the increased usageof such devices in third world countries.

As a result of the rapid pace of development, a relatively highpercentage of electronic devices are replaced every year as consumerscontinually upgrade their mobile phones and other electronic devices toobtain the latest features or a better operating plan. According to theU.S. Environmental Protection Agency, the U.S. alone disposes of over370 million mobile phones, PDAs, tablets, and other electronic devicesevery year. Millions of other outdated or broken mobile phones and otherelectronic devices are simply tossed into junk drawers or otherwise keptuntil a suitable disposal solution arises.

Although many electronic device retailers and cell carrier stores nowoffer mobile phone trade-in or buyback programs, many old mobile phonesstill end up in landfills or are improperly disassembled and disposed ofin developing countries. Unfortunately, however, mobile phones andsimilar devices typically contain substances that can be harmful to theenvironment, such as arsenic, lithium, cadmium, copper, lead, mercury,and zinc. If not properly disposed of, these toxic substances can seepinto groundwater from decomposing landfills and contaminate the soilwith potentiality harmful consequences for humans and the environment.

As an alternative to retailer trade-in or buyback programs, consumerscan now recycle and/or sell their used mobile phones using self-servicekiosks located in malls, retail stores, or other publicly accessibleareas. Such kiosks are operated by ecoATM, LLC, the assignee of thepresent application, and are disclosed in, for example, U.S. Pat. Nos.8,463,646, 8,423,404, 8,239,262, 8,200,533, 8,195,511, and 7,881,965,which are commonly owned by ecoATM, LLC and are incorporated herein byreference in their entireties.

It is often necessary to visually evaluate the physical and/or cosmeticcondition of an electronic device. For example, pricing the electronicdevice, assessing the electronic device for possible repair, andevaluating the electronic device for warranty coverage all can requireidentification of scratches, cracks, water damage, or other cosmeticdefects in the device's screen and/or in non-screen portions of thedevice. Individualized manual inspection of devices can be slow,cumbersome, and can yield inconsistent results among devices. Thereremains a need for more efficient technologies for evaluating thephysical and/or cosmetic condition of electronic devices.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1A is a schematic illustration of a representative operatingenvironment having elements configured in accordance with someembodiments of the present technology.

FIGS. 1B-1E are a series of isometric views of the kiosk shown in FIG.1A with the housing removed to illustrate selected internal componentsconfigured in accordance with some embodiments of the presenttechnology.

FIG. 2 is a flowchart illustrating a method for evaluating the cosmeticcondition of electronic devices in accordance with some embodiments ofthe present technology.

FIG. 3 illustrates an example neural network that can be implemented inaccordance with some embodiments of the present technology.

FIG. 4A illustrates examples of pre-processed images showing a frontside of a smartphone in accordance with some embodiments of the presenttechnology.

FIG. 4B illustrates other examples of pre-processed images showing afront side of a smartphone in accordance with some embodiments of thepresent technology.

FIG. 5 is a flowchart illustrating a method for training a neuralnetwork for evaluating the cosmetic condition of electronic devices inaccordance with some embodiments of the present technology.

FIG. 6 is a block diagram illustrating an example architecture for acomputer system that can be utilized to implement various portions ofthe present technology.

FIG. 7 is a flowchart representation of a method for evaluating aphysical condition of an electronic device in accordance with someembodiments of the present technology.

FIG. 8 illustrates an example architecture of a system for examiningconsumer devices and providing offer prices in accordance with someembodiments of the present technology.

FIG. 9A illustrates a side view of an example arrangement of lightsources 901 a,b in the upper chamber in accordance with one or moreembodiments of the present technology.

FIG. 9B illustrates an example arrangement of two sets of light sourcesin accordance with some embodiments of the present technology.

FIG. 10 illustrates an example of evaluating an electronic device usinganother mobile device in accordance with some embodiments of the presenttechnology.

FIG. 11 illustrates an example architecture of training a neural networkin accordance with some embodiments of the present technology.

DETAILED DESCRIPTION

The present disclosure describes various embodiments of systems andmethods for evaluating the cosmetic and/or physical condition of mobilephones and/or other electronic devices using machine learningtechniques. As described in greater detail below, in some embodimentsthese systems and methods can be implemented by a consumer operatedkiosk to evaluate whether, for example, a display screen of a mobilephone is cracked or otherwise damaged.

Efficiently and consistently evaluating the cosmetic condition ofelectronic devices can be challenging. For example, manualidentification of defects as shown in images of electronic devices canbe costly, tedious, and subject to variability among differentinspectors or even the same inspector. The manual process can also beinaccurate in many cases. For example, when the screen of the device ison, a human inspector can not be able to differentiate cosmetic defectsfrom the background image shown on the device. As another example, ascreen protector or case attached to the device can make manualinspection difficult. In this regard, certain feature- or rule-basedautomatic pattern recognition methods can not provide satisfactory andconsistent evaluation results either. Additionally, the evaluation ofcosmetic condition can not be limited to the identification of apre-defined set of defects (e.g., scratches, cracks, dents, waterdamage, and/or bad pixels). Rather, the evaluation can correspond to acomprehensive, overall “look and feel” of an electronic device, such asidentifying whether a device is a counterfeit product. Therefore,predefined feature- or rule-based methods can be inefficient and/orinsufficient to handle various cosmetic evaluation scenarios.

Aspects of the present technology use machine learning techniques(artificial neural networks (ANNs) in particular) to perform cosmeticcondition evaluation based on images of electronic devices, withoutpredetermined feature(s) or rule(s). Among other things, the use ofANN(s) as described herein contributes to various advantages andimprovements (e.g., in computational efficiency, detection accuracy,system robustness, etc.) in processing images of electronic devices. Asthose skilled in the art would appreciate, ANNs are computing systemsthat “learn” (i.e., progressively improve performance on) tasks byconsidering examples, generally without task-specific programming. Forexample, in image recognition, ANN can learn to identify images thatcontain cats by analyzing example images that have been manually labeledas “cat” or “no cat” and using the results to identify cats in otherimages.

An ANN is typically based on a collection of connected units or nodescalled artificial neurons. Each connection between artificial neuronscan transmit a signal from one artificial neuron to another. Theartificial neuron that receives the signal can process it and thensignal artificial neurons connected to it. Typically, in ANNimplementations, the signal at a connection between artificial neuronsis a real number, and the output of each artificial neuron is calculatedby a non-linear function of the sum of its inputs. Artificial neuronsand connections typically have a weight that adjusts as learningproceeds. The weight increases or decreases the strength of the signalat a connection. Artificial neurons can have a threshold such that onlyif the aggregate signal crosses that threshold is the signal sent.Typically, artificial neurons are organized in layers. Different layerscan perform different kinds of transformations on their inputs. Signalstravel from the first (input) to the last (output) layer, possibly aftertraversing the layers multiple times.

In some embodiments, one or more ANNs used by the present technologyincludes convolutional neural network(s) (CNN or ConvNet). Typically,CNNs use a variation of multilayer perceptrons designed to requireminimal pre-processing. CNNs can also be shift invariant or spaceinvariant artificial neural networks (SIANN), based on theirshared-weights architecture and translation invariance characteristics.Illustratively, CNNs were inspired by biological processes in that theconnectivity pattern between neurons resembles the organization of theanimal visual cortex. Individual cortical neurons respond to stimulionly in a restricted region of the visual field known as the receptivefield. The receptive fields of different neurons partially overlap suchthat they cover the entire visual field.

FIGS. 1A-E illustrate details about a kiosk model in accordance withsome embodiments of the present technology. FIG. 1A illustrates anexample kiosk 100 for recycling, selling, and/or other processing ofmobile phones and other consumer electronic devices in accordance withsome embodiments of the present technology. In some embodiments, atleast some portions of the technology described herein can be carriedout using a kiosk that includes an imaging device therein. For example,the kiosk can process and evaluate images received from the imagingdevice. The kiosk can include, for example, a processing component(e.g., including one or more physical processors) and memory storinginstructions that, when executed by the processing component, perform atleast some operations described herein. The term “processing” is usedherein for ease of reference to generally refer to all manner ofservices and operations that can be performed or facilitated by thekiosk 100 on, with, or otherwise in relation to an electronic device.Such services and operations can include, for example, selling,reselling, recycling, donating, exchanging, identifying, evaluating,pricing, auctioning, decommissioning, transferring data from or to,reconfiguring, refurbishing, etc., mobile phones and other electronicdevices. Although many embodiments of the present technology aredescribed herein in the context of mobile phones, aspects of the presenttechnology are not limited to mobile phones and can generally apply toother consumer electronic devices. Such devices include, as non-limitingexamples, all manner of mobile phones; smartphones; handheld devices;personal digital assistants (PDAs); MP3 or other digital music players;tablet, notebook, ultrabook, and laptop computers; e-readers all types;GPS devices; set-top boxes; universal remote controls; wearablecomputers; etc. In some embodiments, it is contemplated that the kiosk100 can facilitate selling and/or otherwise processing larger consumerelectronic devices, such as desktop computers, TVs, game consoles, etc.,as well smaller electronic devices such as Google® Glass™, smartwatches(e.g., the Apple Watch™, Android Wear™ devices such as the Moto 360®, orthe Pebble Steel™ watch), etc. The kiosk 100 and various featuresthereof can be at least generally similar in structure and function tothe systems, methods and corresponding features described in thefollowing patents and patent applications, which are incorporated hereinby reference in their entireties: U.S. Pat. Nos. 10,127,647, 10,055,798;10,032,140; 9,904,911; 9,881,284; 8,200,533; 8,195,511; 8,463,646;8,423,404; 8,239,262; 8,200,533; 8,195,511; and 7,881,965; U.S. patentapplication Ser. Nos. 12/573,089; 12/727,624; 13/113,497; 12/785,465;13/017,560; 13/438,924; 13/753,539; 13/658,825; 13/733,984; 13/705,252;13/487,299; 13/492,835; 13/562,292; 13/658,828; 13/693,032; 13/792,030;13/794,814; 13/794,816; 13/862,395; 13/913,408; U.S. patent applicationSer. No. 14/498,763, titled “METHODS AND SYSTEMS FOR PRICING ANDPERFORMING OTHER PROCESSES ASSOCIATED WITH RECYCLING MOBILE PHONES ANDOTHER ELECTRONIC DEVICES,” filed by the applicant on Sep. 26, 2014; U.S.patent application Ser. No. 14/500,739, titled “MAINTAINING SETS OFCABLE COMPONENTS USED FOR WIRED ANALYSIS, CHARGING, OR OTHER INTERACTIONWITH PORTABLE ELECTRONIC DEVICES,” filed by the applicant on Sep. 29,2014; U.S. patent application Ser. No. 14/873,158, titled“WIRELESS-ENABLED KIOSK FOR RECYCLING CONSUMER DEVICES,” filed by theapplicant on Oct. 1, 2015; U.S. patent application Ser. No. 14/506,449,titled “SYSTEM FOR ELECTRICALLY TESTING MOBILE DEVICES AT ACONSUMER-OPERATED KIOSK, AND ASSOCIATED DEVICES AND METHODS,” filed bythe applicant on Oct. 3, 2014; U.S. patent application Ser. No.14/925,357, titled “SYSTEMS AND METHODS FOR RECYCLING CONSUMERELECTRONIC DEVICES,” filed by the applicant on Oct. 28, 2015; U.S.patent application Ser. No. 14/925,375, titled “METHODS AND SYSTEMS FORFACILITATING PROCESSES ASSOCIATED WITH INSURANCE SERVICES AND/OR OTHERSERVICES FOR ELECTRONIC DEVICES,” filed by the applicant on Oct. 28,2015; U.S. patent application Ser. No. 14/934,134, titled “METHODS ANDSYSTEMS FOR EVALUATING AND RECYCLING ELECTRONIC DEVICES,” filed by theapplicant on Nov. 5, 2015; U.S. patent application Ser. No. 14/964,963,titled “METHODS AND SYSTEMS FOR PROVIDING INFORMATION REGARDINGCOUPONS/PROMOTIONS AT KIOSKS FOR RECYCLING MOBILE PHONES AND OTHERELECTRONIC DEVICES,” filed by the applicant on Dec. 10, 2015; U.S.patent application Ser. No. 14/568,051, titled “METHODS AND SYSTEMS FORIDENTIFYING MOBILE PHONES AND OTHER ELECTRONIC DEVICES,” filed by theapplicant on Dec. 11, 2014; U.S. patent application Ser. No. 14/966,346,titled “SYSTEMS AND METHODS FOR RECYCLING CONSUMER ELECTRONIC DEVICES,”filed by the applicant on Dec. 11, 2015; U.S. patent application Ser.No. 14/598,469, titled “METHODS AND SYSTEMS FOR DYNAMIC PRICING ANDPERFORMING OTHER PROCESSES ASSOCIATED WITH RECYCLING MOBILE PHONES ANDOTHER ELECTRONIC DEVICES,” filed by the applicant on Jan. 16, 2015; U.S.patent application Ser. No. 14/660,768, titled “SYSTEMS AND METHODS FORINSPECTING MOBILE DEVICES AND OTHER CONSUMER ELECTRONIC DEVICES WITH ALASER,” filed by the applicant on Mar. 17, 2015; U.S. patent applicationSer. No. 14/663,331, titled “DEVICE RECYCLING SYSTEMS WITH FACIALRECOGNITION,” filed by the applicant on Mar. 19, 2015; U.S. provisionalapplication No. 62/169,072, titled “METHODS AND SYSTEMS FOR VISUALLYEVALUATING ELECTRONIC DEVICES,” filed by the applicant on Jun. 1, 2015;U.S. provisional application No. 62/202,330, titled “METHODS AND SYSTEMSFOR INSPECTING MOBILE DEVICES AND OTHER CONSUMER ELECTRONIC DEVICES WITHROBOTIC ACTUATION,” filed by the applicant on Aug. 7, 2015; and U.S.patent application Ser. No. 15/057,707, titled “METHODS AND SYSTEMS FORINTERACTIONS WITH A SYSTEM FOR PURCHASING MOBILE PHONES AND OTHERELECTRONIC DEVICES,” filed by the applicant on Mar. 1, 2016; U.S. patentapplication Ser. No. 15/176,975, titled “METHODS AND SYSTEMS FORDETECTING SCREEN COVERS ON ELECTRONIC DEVICES,” filed by the applicanton Jun. 8, 2016. In some embodiments, the kiosk 100 can share many orall of the features of the kiosks disclosed and described in U.S. patentapplication Ser. No. 16/719,699, entitled “SYSTEMS AND METHODS FORVENDING AND/OR PURCHASING MOBILE PHONES AND OTHER ELECTRONIC DEVICES,”filed on Dec. 18, 2019, U.S. patent application Ser. No. 16/788,169,entitled KIOSK FOR EVALUATING AND PURCHASING USED ELECTRONIC DEVICES,filed on Feb. 11, 2020, U.S. patent application Ser. No. 16/788,153,entitled “CONNECTOR CARRIER FOR ELECTRONIC DEVICE KIOSK,” filed on Feb.11, 2020, and U.S. Provisional Application No. 62/950,075, entitled“SYSTEMS AND METHODS FOR VENDING AND/OR PURCHASING MOBILE PHONES ANDOTHER ELECTRONIC DEVICES,” filed on Dec. 18, 2019. All the patents andpatent applications listed in the preceding sentences and any otherpatents or patent applications identified herein are incorporated hereinby reference in their entireties.

In the illustrated embodiment, the kiosk 100 is a floor-standingself-service kiosk configured for use by a user 101 (e.g., a consumer,customer, etc.) to recycle, sell, and/or perform other operations with amobile phone or other consumer electronic device. In other embodiments,the kiosk 100 can be configured for use on a countertop or a similarraised surface. Although the kiosk 100 is configured for use byconsumers, in various embodiments the kiosk 100 and/or various portionsthereof can also be used by other operators, such as a retail clerk orkiosk assistant to facilitate the selling or other processing of mobilephones and other electronic devices.

In the illustrated embodiment, the kiosk 100 includes a housing 102 thatis approximately the size of a conventional vending machine. The housing102 can be of conventional manufacture from, for example, sheet metal,plastic panels, etc. A plurality of user interface devices is providedon a front portion of the housing 102 for providing instructions andother information to users, and/or for receiving user inputs and otherinformation from users. For example, the kiosk 100 can include a displayscreen 104 (e.g., a liquid crystal display (LCD) or light emitting diode(LED) display screen, a projected display (such as a heads-up display ora head-mounted device), etc.) for providing information, prompts, and soon, to users. The display screen 104 can include a touch screen forreceiving user input and responses to displayed prompts. In someembodiments, the kiosk 100 can include a separate keyboard or keypad forthis purpose. The kiosk 100 can also include an ID reader or scanner 112(e.g., a driver's license scanner), a fingerprint reader 114, and one ormore cameras 116 a-c (e.g., digital still and/or video cameras,identified individually as cameras). The kiosk 100 can additionallyinclude output devices, such as a label printer having an outlet 110,and a cash dispenser having an outlet 118. Although not identified inFIGS. 1A-1E, the kiosk 100 can further include a speaker and/or aheadphone jack for audibly communicating information to users, one ormore lights for visually communicating signals or other information tousers, a handset or microphone for receiving verbal input from the user,a card reader (e.g., a credit/debit card reader, loyalty card reader,etc.), a receipt or voucher printer and dispenser, as well as other userinput and output devices. The input devices can include a touchpad,pointing device such as a mouse, joystick, pen, game pad, motion sensor,scanner, eye direction monitoring system, etc. Additionally, the kiosk100 can also include a bar code reader, QR code reader, bag/packagedispenser, a digital signature pad, etc. In the illustrated embodiment,the kiosk 100 additionally includes a header 120 having a display screen122 for displaying marketing advertisements and/or other video orgraphical information to attract users to the kiosk. In addition to theuser interface devices described above, the front portion of the housing102 also includes an access panel or door 106 located directly beneaththe display screen 104. The access door can be configured toautomatically retract so that the user 101 can place an electronicdevice (e.g., a mobile phone) in an inspection area 108 for automaticinspection, evaluation, and/or other processing by the kiosk 100.

A sidewall portion of the housing 102 can include a number ofconveniences to help users recycle or otherwise process their mobilephones. For example, in the illustrated embodiment the kiosk 100includes an accessory bin 128 that is configured to receive mobiledevice accessories that the user wishes to recycle or otherwise disposeof. Additionally, the kiosk 100 can provide a free charging station 126with a plurality of electrical connectors 124 for charging a widevariety of mobile phones and other consumer electronic devices.

FIGS. 1B-1E illustrate a series of isometric views of the kiosk 100 withthe housing 102 removed to illustrate selected internal componentsconfigured in accordance with some embodiments of the presenttechnology. Referring first to FIG. 1B, in the illustrated embodimentthe kiosk 100 includes a connector carrier 140 and an inspection plate144 operably disposed behind the access door 106 as shown in FIG. 1A. Inthe illustrated embodiment, the connector carrier 140 is a rotatablecarrousel that is configured to rotate about a generally horizontal axisand carries a plurality of electrical connectors 142 (e.g.,approximately 25 connectors) distributed around an outer peripherythereof. In other embodiments, other types of connector carrying devices(including both fixed and movable arrangements) can be used. In someembodiments, the connectors 142 includes a plurality of interchangeableUSB connectors configured to provide power and/or exchange data with avariety of different mobile phones and/or other electronic devices. Inoperation, the connector carrier 140 is configured to automaticallyrotate about its axis to position an appropriate one of the connectors142 adjacent to an electronic device, such as a mobile phone 150, thathas been placed on the inspection plate 144 for recycling. The connector142 can then be manually and/or automatically withdrawn from theconnector carrier 140 and connected to a port on the mobile phone 150for electrical analysis. Such analysis can include, for example, anevaluation of the make, model, configuration, condition, etc.

In the illustrated embodiment, the inspection plate 144 is configured totranslate back and forth (on, e.g., parallel mounting tracks) to move anelectronic device, such as the mobile phone 150, between a firstposition directly behind the access door 106 and a second positionbetween an upper chamber 130 and an opposing lower chamber 132.Moreover, in this embodiment the inspection plate 144 is transparent, orat least partially transparent (e.g., formed of glass, Plexiglas, etc.)to enable the mobile phone 150 to be photographed and/or otherwiseoptically evaluated from all, or at least most viewing angles (e.g.,top, bottom, sides, etc.) using an imaging device 190 (e.g., one or morecameras) mounted to or otherwise associated with the upper and lowerchambers 130 and 132. When the mobile phone 150 is in the secondposition, the upper chamber 130 can translate downwardly to generallyenclose the mobile phone 150 between the upper chamber 130 and the lowerchamber 132. The upper chamber 130 is operably coupled to a gate 138that moves up and down in unison with the upper chamber 130.

In some embodiments, the imaging device 190 can include one or morecameras disposed within both the upper chamber 130 and the lower chamber132 to capture images of top and bottom surfaces of the mobile device150 in order to detect cracks and/or scratches in the screen. The upperchamber 130 and/or the lower chamber 132 can include one or more lightsources (e.g., spotlights) to allow the imaging device 190 to capturehigh quality images that demonstrate cosmetic defects on the mobiledevice 150.

In some embodiments, the one or more light sources are arranged in theupper chamber 130 and/or the lower chamber 132. FIG. 9A illustrates aside view of an example arrangement of light sources 901 a,b in theupper chamber in accordance with one or more embodiments of the presenttechnology. The light beams 911 a,b from the light sources 901 a,b formsmall angles (e.g., equal to or smaller than 60 degrees) with respect tothe display of the mobile phone 150 to avoid direct reflection of thelights from the highly reflective display of the mobile phone 150. Therelative positions between the one or more light sources 901 a,b and theone or more cameras 921 a,b of the imaging device 190 can be adjusted toensure that the reflected light beams 913 a,b from the mobile phone 150can reach the cameras 921 a,b. In some embodiments, kiosks can performself-calibration to adjust the angles of the light sources to ensurethat the correct angles are formed. In some embodiments, technicians canbe dispatched periodically or upon request to perform calibrations ofthe kiosks.

In some embodiments, the one or more light sources includes two sets oflight sources that are arranged orthogonal to each other. Because thecracks and/or scratches on the mobile device 150 can run in differentdirections (e.g., both horizontally and/or vertically), having two setsof orthogonally arranged light sources allows the cameras to capturevarious combinations of the cracks and/or scratches. For example, afirst angle between light beams from one set of lights and the top sideof the inspection plate 144 can be between 30 to 60 degrees (e.g.,preferably 45 degrees) while a second angle between light beams from asecond set of lights and the left side of the inspection plate 144 canbe between 30 to 60 degrees (e.g., preferably 45 degrees). The two setsof lights are positioned orthogonal to each other. FIG. 9B illustratesan example arrangement of two sets of light sources in accordance withsome embodiments of the present technology. A first set of light sources931 a,b is arranged orthogonally with a second set of light sources 941a,b. Light beams 951 a,b from the first set of light sources 931 a,b areabout 45 degrees from either side of the inspection plate 144 (e.g., Xaxis and/or Y axis). Similarly, light beams 961 a,b from the second setof light sources 941 a,b are about 45 degrees from either side of theinspection plate 144 (e.g., X axis or/or y axis). Such arrangement canhelp reduce or eliminate imaging noise or shadows from other componentsof the kiosk 100 that are arranged along the sides of the inspectionplate. In some embodiments, additional sets of light sources can bearranged within the upper and/or lower chamber to reveal damage that cannot be visible from orthogonal arrangements of the light sources.

In some embodiments, the light beams from the one or more light sourcescan be collimated to produce more defined shadows of the cracks and/orscratches. In some embodiments, the one or more light sources support awide range of brightness so that multiple sets of images can be taken atdifferent light intensities with exposure times. For example, differentdevices can have different background colors (e.g., a white phone or ablack phone), which can affect the processing of the captured images.Having at least two sets of images taken at different camera exposures,different light intensities, and/or different white balance settings canallow more accurate processing of cosmetic features of the device.

Because the mobile phone 150 is positioned on the transparent plate 144,light beams from light sources disposed in the lower chamber 132 undergoadditional reflections within the transparent plate 144 before reachingthe mobile phone 150, thereby impacting the quality of the capturedimages. Therefore, in some embodiments, all cameras of the imagingdevice 190 and the light sources are disposed within the upper chamber130 only. The kiosk 100 can include a flipping mechanism 148 (e.g., arobot arm) to flip the mobile phone 150 so that images of both the topand bottom surfaces of the mobile phone 150 can be captured without anyreflections between the cameras and the mobile phone 150.

Furthermore, to improve quality of the captured images, the color of theupper chamber 130 and the lower chamber 132 can be a middle gray, suchas the 18% gray for calibrating light meters. A proper color of thechambers provides enough contract for glints over the display andshadows of hairline cracks of the mobile phone 150.

The images captured by the kiosk 100 can be transmitted to a qualifiedhuman operator to examine the quality of images as a measure of ensuringinput quality to the computer-implemented visual analysis.Alternatively, the captured images can be transmitted to another neuralnetwork model to automatically determine the quality of the images andto provide feedback to the kiosks. If an operator or the neural networkmodel determines that images captured by a particular kiosk routinelydemonstrate certain defects (e.g., images are too dark, images areoverexposed, etc.), technicians can be dispatched to re-calibrate thekiosk to ensure that uniform input images are obtained at differentkiosks.

In some embodiments, the upper chamber 130 and/or the lower chamber 132can also include one or more magnification tools, scanners (e.g., barcode scanners, infrared scanners, etc.), or other imaging components(not shown) and an arrangement of mirrors (also not shown) to view,photograph, and/or otherwise visually evaluate the mobile phone 150 frommultiple perspectives. In some embodiments, one or more of the camerasand/or other imaging components discussed above can be movable tofacilitate device evaluation. For example, as noted above with respectto FIG. 1A, the imaging device 190 can be affixed to a moveablemechanical component, such as an arm, which in turn can be moved using abelt drive, rack and pinion system, or other suitable drive systemcoupled to an electronic controller (e.g., the computing device). Theinspection area 108 can also include weight scales, heat detectors, UVor infrared readers/detectors, and the like, for further evaluation ofelectronic devices placed therein. For example, information from theweight scales, UV, or infrared readers/detections can provide accurateinformation to facilitate the determination of the model of the mobilephone 150. The kiosk 100 can further include an angled binning plate 136for directing electronic devices from the transparent plate 144 into acollection bin 134 positioned in a lower portion of the kiosk 100.

The kiosk 100 can be used in a number of different ways to efficientlyfacilitate the recycling, selling, and/or other processing of mobilephones and other consumer electronic devices. Referring to FIGS. 1A-1Etogether, in one embodiment, a user 101 wishing to sell a used mobilephone, such as the mobile phone 150, approaches the kiosk 100 andidentifies the type of device (e.g., a mobile phone, a tablet, etc.) theuser wishes to sell in response to prompts on the display screen 104.Next, the user can be prompted to remove any cases, stickers, or otheraccessories from the device so that it can be accurately evaluated.Additionally, the kiosk 100 can print and dispense a uniqueidentification label (e.g., a small adhesive-backed sticker with a quickresponse code (“QR code”), barcode, or other machine-readable indicia,etc.) from the label outlet 110 for the user to adhere to the back ofthe mobile phone 150. After this is done, the door 106 retracts andopens allowing the user to place the mobile phone 150 onto thetransparent plate 144 in the inspection area 108 as shown in FIG. 1B.The door 106 then closes and the transparent plate 144 moves the mobilephone 150 under the upper chamber 130 as shown in FIG. 1C. The upperchamber 130 then moves downwardly to generally enclose the mobile phone150 between the upper and lower chambers 130 and 132, and the camerasand/or other imaging components in the upper and lower chambers 130 and132 perform a visual inspection of the mobile phone 150. In someembodiments, the visual inspection of the mobile phone 150 includesperforming at least a part of method 200 (as shown in FIG. 2), at leasta part of method 500 (as shown in FIG. 5), and/or at least a part ofmethod 600 (as shown in FIG. 6) to evaluate the physical and/or cosmeticcondition of the mobile phone 150. In some embodiments, the visualinspection includes a computer-implemented visual analysis (e.g., athree-dimensional (3D) analysis) performed by a processing device withinthe kiosk to confirm the identification of the mobile phone 150 (e.g.,make, model, and/or sub-model) and/or to evaluate or assess thecondition and/or function of the mobile phone 150 and/or its variouscomponents and systems. For example, the visual analysis can includecomputer-implemented evaluation (e.g., a digital comparison) of imagesof the mobile phone 150 taken from top, side, and/or end viewperspectives to determine length, width, and/or height (thickness)dimensions of the mobile phone 150. The visual analysis can furtherinclude a computer-implemented inspection of a display screen and/orother surface of the mobile phone 150 to check for, for example, cracksin the glass and/or other damage or defects in the LCD (e.g., defectivepixels, etc.).

Referring next to FIG. 1D, after the visual analysis is performed andthe device has been identified, the upper chamber 130 returns to itsupper position and the transparent plate 144 returns the mobile phone150 to its initial position near the door 106. The display screen 104can also provide an estimated price, or an estimated range of prices,that the kiosk 100 can offer the user for the mobile phone 150 based onthe visual analysis, and/or based on user input (e.g., input regardingthe type, condition, etc., of the phone 150). If the user indicates(via, e.g., input via the touch screen) that they wish to proceed withthe transaction, the connector carrier 140 automatically rotates anappropriate one of the connectors 142 into position adjacent thetransparent plate 144, and door 106 is again opened. The user can thenbe instructed (via, e.g., the display screen 104) to withdraw theselected connector 142 (and its associated wire) from the carrousel 140,plug the connector 142 into the corresponding port (e.g., a USB port) onthe mobile phone 150, and reposition the mobile phone 150 in theinspection area on the transparent plate 144. After doing so, the door106 once again closes and the kiosk 100 (e.g., the kiosk CPU) performsan electrical inspection of the device via the connector 142 to furtherevaluate the condition of the phone, as well as specific component andoperating parameters, such as the memory, carrier, etc. In someembodiments, the electrical inspection can include a determination ofphone manufacturer information (e.g., a vendor identification number orVID) and product information (e.g., a product identification number orPID). In some embodiments, the kiosk 100 can perform the electricalanalysis using one or more of the methods and/or systems described indetail in the commonly owned patents and patent applications identifiedherein and incorporated by reference in their entireties.

After the visual and electronic analysis of the mobile phone 150, theuser 101 is presented with a phone purchase price via the display screen104. If the user declines the price (via, e.g., the touch screen), aretraction mechanism (not shown) automatically disconnects the connector142 from the mobile phone 150, the door 106 opens, and the user canreach in and retrieve the mobile phone 150. If the user accepts theprice, the door 106 remains closed and the user can be prompted to placehis or her identification (e.g., a driver's license) in the ID scanner112 and provide a thumbprint via the fingerprint reader 114. As a fraudprevention measure, the kiosk 100 can be configured to transmit an imageof the driver's license to a remote computer screen, and an operator atthe remote computer can visually compare the picture (and/or otherinformation) on the driver's license to an image of the person standingin front of the kiosk 100 as viewed by one or more of the cameras 116a-c as shown in FIG. 1A to confirm that the person attempting to sellthe phone 150 is in fact the person identified by the driver's license.In some embodiments, one or more of the cameras 116 a-c can be movableto facilitate viewing of kiosk users, as well as other individuals inthe proximity of the kiosk 100. Additionally, the person's fingerprintcan be checked against records of known fraud perpetrators. If either ofthese checks indicate that the person selling the phone presents a fraudrisk, the transaction can be declined and the mobile phone 150 returned.After the user's identity has been verified, the transparent plate 144moves back toward the upper and lower chambers 130 and 132. As shown inFIG. 1E, when the upper chamber 130 is in the lower position, the gate138 permits the transparent plate 144 to slide underneath but notelectronic devices carried thereon. As a result, the gate 138 knocks themobile phone 150 off of the transparent plate 144, onto the binningplate 136 and into the bin 134. The kiosk can then provide payment ofthe purchase price to the user. In some embodiments, payment can be madein the form of cash dispensed from the cash outlet 118. In otherembodiments, the user can receive remuneration for the mobile phone 150in various other useful ways. For example, the user can be paid via aredeemable cash voucher, a coupon, an e-certificate, a prepaid card, awired or wireless monetary deposit to an electronic account (e.g., abank account, credit account, loyalty account, online commerce account,mobile wallet etc.), Bitcoin, etc.

As those of ordinary skill in the art will appreciate, the foregoingroutines are but some examples of ways in which the kiosk 100 can beused to recycle or otherwise process consumer electronic devices such asmobile phones. Although the foregoing example is described in thecontext of mobile phones, it should be understood that the kiosk 100 andvarious embodiments thereof can also be used in a similar manner forrecycling virtually any consumer electronic device, such as MP3 players,tablet computers, PDAs, and other portable devices, as well as otherrelatively non-portable electronic devices, such as desktop computers,printers, devices for implementing games, entertainment or other digitalmedia on CDs, DVDs, Blu-ray, etc. Moreover, although the foregoingexample is described in the context of use by a consumer, the kiosk 100in various embodiments thereof can similarly be used by others, such asa store clerk, to assist consumers in recycling, selling, exchanging,etc., their electronic devices.

FIG. 8 illustrates an example architecture of a system 800 for examiningconsumer devices and providing offer prices in accordance with someembodiments of the present technology. The system 800 includes acapturing module 801 that captures information about consumer devices.The capturing module 801 can be implemented on a kiosk as described inconnection with FIGS. 1A-1E. The capturing module 801 can capture deviceinformation 811 such as the device identifier (ID) of a consumer device,the time and/or the location that the consumer device is examined. Thecapturing module 801 can also capture images 813 of various surfaces ofthe device that demonstrate various features, such as cosmetic defect(s)of the consumer device, which can indicate the condition of the device.For example, images can be captured to show sides of the device,location or existence of buttons on the device, light emitted from thescreen to indicate the LCD panel health. In some embodiments, images canbe captured while the device is moving so as to capture the nature andextent of the damage. The images can also show depth of scratches and/orcracks to facilitate an estimation of the impact to underlyingelectronics. In some embodiments, the entire system 800 can beimplemented on the kiosk 100.

The input information captured by the capturing module 801 istransmitted to a price prediction model 803 that is configured todetermine candidate price for the input consumer device. The priceprediction model 805 can extract features (e.g., scratches, hairlinecracks, water damage marks) from the input information and determine thecandidate price based on the number of cosmetic defects on the device.Alternatively and/or additionally, the capturing module 801 can extractfeatures from the input information and transmit the extracted featuresto the price prediction model 803 to determine the candidate price basedon the number of cosmetic defects on the device.

The system 800 also includes a pricing policy model 805 that acceptsinput from both the capturing module 801 and the price prediction model805. The pricing policy model 805 can leverage various sub-models togenerate a final offer price. The sub-models can include at least asub-model to predict resale value, a sub-model to predict incomingvolume of the consumer device, a sub-model to predict processing costsassociated with the device, and/or other sub-models to facilitate theprediction process. Additional features that can affect the final offerprice include the location of kiosk, the time at which the device wasexamined, the age of the device, the predicted repair costs, volume ofdevices in similar conditions, risk of counterfeit or fraud, theanticipated demand of the device, predicted resale channels, otherelectrical information retrieved from the device. These sub-models canlocate centrally with the pricing policy model. The sub-models can alsobe distributed across different locations in a network as part of acloud-based computing service. Each of the models and/or sub-models canbe implemented using a neural network, such as CNN and/or ConvNet. Ascompared to human operators, the neural networks can produce moreconsistent analysis results across different geographical locations andare much more scalable when a large number of consumer devices need tobe evaluated.

Upon customer's acceptance or rejection of the final offer price, therelevant data for this consumer device can be fed back to the priceprediction model for further training and improvement of the model. Asmentioned above, the capturing module 801 can be deployed in a kioskwhile the other parts of the system are situated in a distributed mannerin remote server(s). In some embodiments, the entire system can bedeployed in a kiosk as described in detail in connection with FIGS.1A-1E.

In some embodiments, instead of finding a kiosk to perform evaluation ofa used consumer device (as discussed in connection with FIGS. 1A-E), thecustomer can download and install a software implementation of thecapturing module 801 on another device (e.g., another mobile phone,tablet, wearable device, and so on). The software implementation of thecapturing module 801 can provide a user interface to the customer tospecify device information 811 (e.g., device ID, brand, model, etc.) andto capture images 813 of the target consumer device. FIG. 10 illustratesan example of evaluating an electronic device 1005 using another mobiledevice 1003 in accordance with some embodiments of the presenttechnology. A customer 1001 can download a software application on hiscurrent mobile device 1003 (also referred to as the capturing device).The software application is configured to control one or more of a lightsource (e.g., a flash light) and/or camera(s) of the mobile device 1003to capture at least one image of a target electronic device 1005. Thecustomer 1001 can also be prompted to provide additional informationabout the target device 1005, such as device manufacturer, model,purchase date, general condition(s), device features, etc., via a userinterface.

Referring back to FIG. 8, the input data (e.g., the captured imagesand/or additional device information provided by the customer) can betransmitted over a network to remote server(s) that host the priceprediction model 803 and the pricing policy model 805 to determine thecondition of the target device and/or a final offer price. Once thefinal offer price is determined, the capturing module 801 can displaythe final offer price of the target device on a user interface of thecapturing device, and the customer can determine whether to accept orreject the offer price. Upon customer's acceptance or rejection of thefinal offer price, the relevant data for this consumer device can be fedback to the price prediction model 803 for further training of themodel. If the customer accepts the offer price, the capturing module 801can provide further instructions to package and mail the device tocorresponding recycling and processing center(s).

To ensure image quality of the captured images, in some embodiments, thecapturing module 801 can control the light source(s) of the capturingdevice to produce various light conditions. The capturing model 801 canfurther provide a set of predetermined settings or templates to guidethe customer to take images of the target consumer device. Each settingor template can specify at least a desired angle to hold the capturingdevice with respect to the used consumer device, a desired exposurelevel, a desired light intensity, a desired white balance level,brightness, contrast, and/or other parameters. The predeterminedtemplates help users to capture uniform input data to allow the systemto generate consistent analysis results.

In some cases, network bandwidth limit can cause delays when a largeamount of input date (e.g., a large set of images) needs to betransmitted to the remote server(s). To address such problems, some ofthe computation logic (e.g., pre-processing of the captured image) canbe deployed locally on the capturing device. For example, a neuralnetwork that performs feature extraction to extract cosmetic defects(e.g., scratches, cracks, water marks, etc.) can be deployed on thecapturing device as a part of the capturing module. Once the featuresare extracted, only the extracted features and information about thedevice (e.g., device ID, model, release date) are transmitted over thenetwork to the prediction and policy models, thereby reducing bandwidthrequirements for transmitting the relevant data.

In some embodiments, pre-processing of the images also includesoperations, such as filtering, scrubbing, normalization, or the like, togenerate preliminary features as input to feed into the neuralnetwork(s). As discussed above, pre-processing the captured images canalleviate network bandwidth limit for transmitting data in someembodiments. Pre-processing of the images can also be particularlyuseful for capturing modules that are deployed on customers' own devicesbecause, unlike the kiosks, customers generally do not have accuratecontrol of the cameras and positions of the devices. For example,pre-processing can adopt object detection algorithms to remove imagesthat fail to include any consumer devices. Pre-processing of the imagescan also generate uniform inputs that are suitable for visual analysisby the neural networks so as to produce consistent results. For example,based on image segmentation techniques, an image of an electronic devicecan be cropped to show one side (e.g., front, back, top, bottom, or thelike) of the electronic device. For the same device, cropped imagesshowing different sides can be combined into a single image.

FIG. 2 is a flowchart illustrating a method 200 for evaluating thecosmetic condition of electronic devices, in accordance with someembodiments of the present technology. With reference to FIG. 2, themethod includes feeding one or more images of an electronic device to apre-processing module 210. In some embodiments, the image(s) can beobtained by the various camera(s) and/or other imaging component(s) ofthe kiosk 100 as described with reference to FIGS. 1A-1E or a capturingdevice owned by the customer. As describe above, the image(s) can bepre-processed to generate preliminary features. In some embodiments, thepre-processing can be performed by the processing component of the kiosk100 or by the capturing device. In other embodiments, the image(s) canbe transmitted to a remote system or device (e.g., a cloud-basedcomputing service), and at least some or all of the pre-processingoperations can be performed remotely. Illustratively, an image of anelectronic device can be cropped to show one side (e.g., front, back,top, bottom, or the like) of the electronic device. Alternatively or inaddition, the images can be taken under natural and/or controlledlighting. Still further, the images can be taken while the device ispowered on or off. For the same device, cropped images showing differentsides, images taken under different lighting, images taken while thedevice is on or off, and/or images of the device taken with othercontrolled/uncontrolled conditions can be combined into a single image.

The pre-processing can further include resizing an image (either anoriginal image, combined image, or otherwise processed image) to apredefined size. The image is resized to provide a uniform input to thecosmetic evaluation neural network. The predefined size for neuralnetwork input can be determined in a manner that generally does notaffect ability to detect cosmetic defects. For example, the predefinedsize must be sufficiently large so that damages or defects shown in anoriginal image still appear in the resized image. Illustratively, eachimage can be resized to 299×299 pixels. In some embodiments, if theimage is a color image, the present technology can separate out the red,green, and blue color spaces and convert the image into athree-dimensional integer matrix.

In some embodiments, if the image is a color image, the presenttechnology can separate out the various color spaces (e.g., the red,green, and blue color spaces) and convert the image into amulti-dimensional (e.g., three-dimensional) integer matrix. For example,as used in standard RGB encoding, each value in the matrix is an integerranging from 0-255. In some embodiments, the matrix can be rescaled bydividing by 255 to create a decimal value between 0 and 1 for eachmatrix entry.

FIGS. 4A and 4B illustrate examples of pre-processed images 400 a-h forinputting into neural network(s) in accordance with some embodiments ofthe present technology. FIG. 4A illustrates a combined image showing thefront side of a smartphone 402 under three different scenarios: lightingof a first white balance setting with the screen turned on 400 a,lighting of a second white balance setting with the screen turned on 400b, the screen turned off 400 c, and the back side of the smartphone 400d. The images do not show obvious scratches or hairline cracks, thus thesmartphone 402 can be considered as in “cosmetically good” condition.FIG. 4B illustrates a combined image showing the front side of asmartphone 404 under three different scenarios: lighting of the firstwhite balance setting with the screen turned on 400 e, lighting of asecond white balance setting with the screen turned on 400 f, the screenturned off 400 g, and the back side of the smartphone 400 h. Thiscombined image shows scratches on the screen of the smartphone 404, thusthe smartphone 404 can be considered as in “cosmetically bad” condition.

Referring back to FIG. 2, the method 200 includes feeding thepreliminary features 212 (e.g., original image, pre-processed image, orthree-dimensional matrix depending on whether or how pre-processing isperformed) into the neural network(s) 220. The neural network(s) caninclude the price prediction model and pricing policy model as shown inFIG. 1F. The method 200 further includes obtaining output 222 from theneural network(s) 220.

In some embodiments, the output of the neural network(s) includes aninteger 0 or 1. Zero can represent “cosmetically good” (e.g.,non-cracked, without significant scratches, or the like), and 1 canrepresent “cosmetically bad” (e.g., cracked, with significant scratches,or the like). In these embodiments, rescaling the inputs to a rangebetween 0 and 1 can help the network train more consistently, as theinputs and outputs are more closely aligned. In some embodiments,instead of a binary value, the output of the neural network(s) can be ascore of a range of values that indicate the severity of the damages onthe consumer device. The output of the neural network(s) can alsoinclude at least one of a cosmetic rating or a category, a type ofdefect(s) detected, an orientation of defect(s) detected, a location ofdefect(s) detected, a size of defect(s) detected, associated confidencelevel(s), or other cosmetic evaluation indication. In some embodiments,the output of the neural network(s) can further include a brand, model,and/or type of the electronic device shown in the input image.Experimental results have demonstrated that the accuracy of neuralnetwork(s) in determining cosmetic defects can achieve 91%, whichexceeds average human capacity (accuracy around 89.9%).

As discussed above, the neural networks 220 can be implemented as a partof the processing component of the kiosk 100 as described above withreference to FIGS. 1A-E or a user device. In other embodiments, at leastsome portion of the neural networks 220 can be implemented on a remotesystem or device (e.g., a cloud-based computing service). In thesecases, the complete set of input date (e.g., images of the electronicdevice 202), the preliminary features 212, and/or certain intermediatedata (e.g., the input/output between neural network layers) can betransmitted to the remote system or device for processing.

FIG. 3 illustrates an example neural network 300 that can be implementedin accordance with some embodiments of the present technology. Theexample neural network 300 can be a CNN or a modified CNN. The exampleneural network 300 can include two main types of network layers, namely,the convolution layer and the pooling layer. A convolution layer can beused to extract various features from the input to convolution layer. Inparticular, different kernel sizes can be applied in convolution layersfor feature extraction to account for the fact that scratches and/orhairline cracks have various sizes. A pooling layer can be utilized tocompress the features that are input to the pooling layer, therebyreducing the number of training parameters for the neural network andeasing the degree of model over-fitting. The example neural network 300can include multiple cascaded convolution and pooling layers that areconnected with one another in various structural arrangements (e.g.,serial connection). In some embodiments, the final layers of the networkcan include a layer of dense fully connected nodes, a dropout layer tomitigate overfitting, and/or one or more sigmoid activations to derivethe final classification. In some embodiments, a sigmoid activation canbe used for binary prediction (e.g., outputting values 0 and 1indicating whether the condition of the device is acceptable). In someembodiments, other types of activation (e.g., a softmax activation) canbe used so that the neural network can output different categories ofpredictions (e.g., “Fraud-Do Not Buy”, “Fake”, etc).

FIG. 11 illustrates an example architecture 1100 of training a neuralnetwork in accordance with some embodiments of the present technology.As shown in FIG. 11, the neural networks can be trained usingpre-collected images 1101 which have been labeled by inspectors 1103(e.g., human inspectors, electronic labeling systems, etc.). In someembodiments, images in the training set are each associated with acosmetic evaluation indication (e.g., “cosmetically good” or“cosmetically bad”) agreed on by at least a threshold number ofinspectors (e.g., two human inspectors). Therefore, the training setincludes representative images of electronic devices in a particularcosmetic status that a threshold number of inspectors have agreed are,and the cosmetic status can be reasonably determined by visualinspection without requiring presence of the device phone on site.

The training set can include images that have been pre-processed thesame way as would image(s) that contribute to the input of the machinelearning system 1105 (e.g., neural network(s)) once it is deployed. Thetraining set can include equal-sized or substantially equal-sized (e.g.,within 5%, 10%, or 15% difference in size) subsets of images associatedwith each distinct cosmetic evaluation indication. For example, forapproximately 700,000 images used in training, about 350,000 areassociated with a “cosmetically good” indication and the other 350,000are associated with a “cosmetically bad” indication. Dividing thetraining set in this manner can prevent or mitigate certain “randomguess” effects of trained neural network(s), where an output can bebiased to favor those reflected by a larger portion of the training set.In some embodiments, at least some of the images in the training set canbe mirrored, rotated, or subject to other positional processing togenerate additional images for inclusion in the training set.

The trained neural network 1105 can be validated using otherpre-collected images which have been labeled by human inspectors 1103.Similar to the training set, a validation set can include subsets ofimages associated with each distinct cosmetic evaluation indication. Incontrast with the training set, the relative sizes of the subsets aremore consistent or otherwise reflect the real-world statistics ofelectronic devices that have previously been evaluated. Illustratively,approximately 300,000 images are used for validating the trained neuralnetwork.

In some embodiments, the machine learning system 1105 (e.g., neuralnetwork(s)) is deployed after successful validation (e.g., the falsepositive and/or false negative rate of the network's output over thevalidation set does not exceed predefined threshold(s)). Additionaldata, such as a portion of the captured images 1107 to the deployednetwork and/or associated outputs that have been verified by humaninspectors, can be collected for further training of the neural network.In some embodiments, for each round of further training, layers closer(e.g., within a threshold number) to the input layer can be frozen whileparameters of layers closer to the output can be adjusted. Doing so canhelp preserve concrete, basic aspects (e.g., representing smallfractions of cracks in different orientations) already learned by thenetwork while allowing the network to adjust parameters directed to moregeneralized, higher level features, which can efficiently adapt to newermodels of devices, different lightings, and/or other changed scenarios.For example, the concrete, basic features learned when training oncracks for an iPhone 8 can still be applicable for detecting cracks on aGalaxy 9, even if the phones are different in size, shape, color, etc.In some embodiments, as shown in FIG. 11, a portion of the capturedimages can be directed to human inspectors 1103 to perform manualevaluation and/or generate more training data for the machine learningsystem 1105.

FIG. 5 is a flowchart illustrating a method 500 for training a neuralnetwork for evaluating the cosmetic condition of electronic devices inaccordance with some embodiments of the present technology. In variousembodiments, the method 500 can be performed by a remote system ordevice associated with the kiosk 100 as described with reference toFIGS. 1A-1E. With reference to FIG. 5, at block 510, the method 500includes creating a training set including equally sized or similarlysized subsets of images associated with each distinct cosmeticevaluation indication.

The training set can include pre-collected images (e.g., those obtainedby the kiosk 100) which have been labeled by human inspectors. In someembodiments, images in the training set are each associated with acosmetic evaluation indication (e.g., “cosmetically good” or“cosmetically bad”) agreed on by at least two human inspectors. Theimages in the training set can be pre-processed the same way as wouldimage(s) that contribute to the input of the neural network once it isdeployed. The training set can include equal-sized or substantiallyequal-sized (e.g., within 5%, 10%, or 15% difference in size) subsets ofimages associated with each distinct cosmetic evaluation indication. Insome embodiments, at least some of the images in the training set can bemirrored, rotated, or subject to other positional processing to generateadditional images for inclusion in the training set.

In addition, the training set can include information about the devices(e.g., brand, model, release date) so that the model can be trained toidentify damages that are specific to a particular set of devices.

At block 520, the method 500 includes training at least a portion of theneural network based on the training set. Illustratively, the trainingset provides “ground-truth” samples of network input and associatedoutput (e.g., sample image(s) of an electronic device and associatedcosmetic evaluation indication), and the components of the neuralnetwork can be trained in various ways as deemed proper by those skilledin the art. The parameters of the neural network can be learned througha sufficiently large number of training samples in the training set.

At block 530, the method 500 includes creating a validation set,including subsets of images associated with each distinct cosmeticevaluation indication, that are generally consistent in relative size asreflected in real world statistics. Similar to the training set, avalidation set can include subsets of images associated with eachdistinct cosmetic evaluation indication. In contrast with the trainingset, the relative sizes of the subsets can be more consistent orotherwise reflect the real-world statistics of electronic devices thathave previously been evaluated.

At block 540, the method 500 includes validating the trained neuralnetwork, and if successful, deploy the neural network. As describedabove, in some embodiments, each class of output is equally (orsubstantially equally) represented during training, but the ratio amongthe output classes is more consistent with field statistics duringvalidation. Such arrangements can be a basis for determining that thetrained network is not generally classifying every input in a particulardirection (e.g., a particular cosmetic evaluation indication), and canstill effectively extract cosmetic condition(s) that is less representedin the dataset.

The neural network can be deployed (e.g., to be executed on the kiosk100 or as a part of the capturing module on a customer's device) aftersuccessful validation (e.g., the false positive and/or false negativerate of the network's output over the validation set does not exceedpredefined threshold(s)). In some embodiments, the method 500 includescollecting additional data (e.g., inputs to the deployed network andassociated outputs that have been verified by human inspectors) forfurther training of the neural network. This can be achieved by loopingback to block 510 of the method. In some embodiments, for each round offurther training, layers closer (e.g., within a threshold number) to theinput layer can be frozen while parameters of layers closer to theoutput can be adjusted. Doing so can help preserve concrete, basicaspects (e.g., representing small fractions of cracks in differentorientations) already learned by the network while allowing the networkto adjust parameters directed to more generalized, higher levelfeatures, which can efficiently adapt to newer models of devices,different lightings, and/or other changed scenarios.

FIG. 6 is a block diagram illustrating an example of the architecturefor a computer system 600 that can be utilized to implement variousportions of the present technology. In FIG. 6, the computer system 600includes one or more processors 605 and memory 610 connected via aninterconnect 625. The interconnect 625 can represent any one or moreseparate physical buses, point to point connections, or both, connectedby appropriate bridges, adapters, or controllers. The interconnect 625,therefore, can include, for example, a system bus, a PeripheralComponent Interconnect (PCI) bus, a HyperTransport or industry standardarchitecture (ISA) bus, a small computer system interface (SCSI) bus, auniversal serial bus (USB), IIC (I2C) bus, or an Institute of Electricaland Electronics Engineers (IEEE) standard 674 bus, sometimes referred toas “Firewire.”

The processor(s) 605 can include central processing units (CPUs) tocontrol the overall operation of, for example, the host computer. Incertain embodiments, the processor(s) 605 accomplish this by executingsoftware or firmware stored in memory 610. The processor(s) 605 can be,or can include, one or more programmable general-purpose orspecial-purpose microprocessors, digital signal processors (DSPs),programmable controllers, application specific integrated circuits(ASICs), programmable logic devices (PLDs), or the like, or acombination of such devices.

The memory 610 can be or include the main memory of the computer system.The memory 610 represents any suitable form of random access memory(RAM), read-only memory (ROM), flash memory, or the like, or acombination of such devices. In use, the memory 610 can contain, amongother things, a set of machine instructions which, when executed byprocessor(s) 605, causes the processor(s) 605 to perform operations toimplement embodiments of the present technology. In some embodiments,the memory 610 can contain an operating system (OS) 630 that managescomputer hardware and software resources and provides common servicesfor computer programs.

Also connected to the processor(s) 605 through the interconnect 625 is a(optional) network adapter 615. The network adapter 615 provides thecomputer system 600 with the ability to communicate with remote devices,such as the storage clients, and/or other storage servers, and can be,for example, an Ethernet adapter or Fiber Channel adapter.

The techniques described herein can be implemented by, for example,programmable circuitry (e.g., one or more microprocessors) programmedwith software and/or firmware, or entirely in special-purpose hardwiredcircuitry, or in a combination of such forms. Special-purpose hardwiredcircuitry can be in the form of, for example, one or moreapplication-specific integrated circuits (ASICs), programmable logicdevices (PLDs), field-programmable gate arrays (FPGAs), etc. Systemsimplemented using the disclosed techniques can be deployed eithercentrally (e.g., the kiosks) or in a distributed manner (e.g., clientdevice and remote servers) according to network resources, bandwidthcost, desired performance, etc.

Software or firmware for use in implementing the techniques introducedhere can be stored on a machine-readable storage medium and can beexecuted by one or more general-purpose or special-purpose programmablemicroprocessors. A “machine-readable storage medium,” as the term isused herein, includes any mechanism that can store information in a formaccessible by a machine (a machine can be, for example, a computer,network device, cellular phone, personal digital assistant (PDA),manufacturing tool, any device with one or more processors, etc.). Forexample, a machine-accessible storage medium includesrecordable/non-recordable media (e.g., read-only memory (ROM); randomaccess memory (RAM); magnetic disk storage media; optical storage media;flash memory devices; etc.). The term “logic,” as used herein, caninclude, for example, programmable circuitry programmed with specificsoftware and/or firmware, special-purpose hardwired circuitry, or acombination thereof.

FIG. 7 is a flowchart representation of a method 700 for evaluating acondition of an electronic device in accordance with some embodiments ofthe present technology. The method 700 includes, at operation 710,capturing, by at least one camera of a kiosk, at least one image of afirst side of the electronic device, wherein the kiosk includes multiplelight sources. The method 700 includes, at operation 720, extracting, bya neural network, a set of features of the electronic device based onthe at least one image of the electronic device. The method 700 alsoincludes, at operation 830, determining a condition of the electronicdevice based on the set of features.

In some embodiments, the method includes capturing, via the at least onecamera, at least one image of a second side of the electronic devicethat is different from the first side based on at least one lightingcondition generated by the multiple light sources. Using differentsettings of the light sources and/or cameras to create differentlighting conditions can facilitate the imaging of the scratches and/orhairline cracks. To image the second side of the electronic device, themethod can include, prior to capturing the at least one image of thesecond side of the electronic device, flipping the electronic devicesuch that the light beams are directed towards the second side of theelectronic device. In some embodiments, images are pre-processed asdescribed in connection with FIGS. 4A-B. The method includes processingmultiple images of multiple sides of the electronic device such that themultiple images have a uniform size and combining the multiple imagesinto a single image to be provided to the neural network.

The angle of the light beams and the arrangement of the light sourcescan affect the final captured images, as discussed in connection withFIGS. 9A-B. In some embodiments, the method includes adjusting one ofthe multiple light sources such that an angle between a light beam fromthe light source and the first side of the electronic device is equal toor smaller than 60 degrees.

In some embodiments, the method includes determining a model of theelectronic device in part based on the at least one image andidentifying a cosmetic defect on the electronic device that is specificto the model. In some embodiments, the method includes determining, viaa second neural network, an price for the electronic device in partbased on the initial estimated price. The final offer price can bedetermined further based on at least (1) a predicted resale value of theelectronic device, (2) a predicted incoming volume of a model of theelectronic device, or (3) a predicted processing cost of the electronicdevice. In some embodiments, the method includes receiving an input froma user indicating an acceptance or a rejection of the final price andtraining the neural network in part based on the at least one image andthe input from the user.

Some examples of the disclosed techniques are further described below.

Example 1

A system for evaluating a condition of an electronic device, comprising:a kiosk that includes an inspection plate configured to hold theelectronic device, one or more light sources arranged above theinspection plate configured to direct one or more light beams towardsthe electronic device; and one or more cameras configured to capture atleast one image of a first side of the electronic device based on atleast one lighting condition generated by the one or more light sources.The system also includes one or more processors in communication withthe one or more cameras, the one or more processors configured toextract a set of features of the electronic device based on the at leastone image of the electronic device; and determine, via a first neuralnetwork, a condition of the electronic device based on the set offeatures.

Example 2

The system of example 1, wherein the one or more light sources comprisesa first subset of light sources and a second subset of light sources,light beams of the first subset of light sources and light beams of thesecond subset of light sources arranged to be orthogonal to each other.

Example 3

The system of example 1 or 2, wherein the kiosk further includes: anupper chamber positioned above the inspection plate, wherein the one ormore light sources are arranged within the upper chamber; a lowerchamber positioned below the inspection plate, and a second set of lightsources positioned within the lower chamber configured to direct lightbeams towards the electronic device through the inspection plate.

Example 4

The system of one or more of examples 1 to 3, wherein the kiosk furtherincludes: a flipping mechanism configured to flip the electronic deviceto allow the one or more cameras to capture at least another image of asecond side of the electronic device.

Example 5

The system of one or more of examples 1 to 4, wherein at least one ofthe one or more light sources is configured to produce a collimatedlight beam.

Example 6

The system of one or more of examples 1 to 5, wherein an angle between alight beam from one of the one or more light sources and the first sideof the electronic device is equal to or smaller than 60 degrees.

Example 7

The system of one or more of examples 1 to 6, wherein the one or morecameras are configured to capture multiple images corresponding tomultiple sides of the electronic devices under different lightingconditions, and wherein the one or more processors are configured toprocess and combine the multiple images into a single input image.

Example 8

The system of one or more of examples 1 to 7, wherein the first neuralnetwork is configured to output an indicator indicating the condition ofthe electronic device.

Example 9

The system of one or more of examples 1 to 8, wherein the one or moreprocessors are further configured to determine an estimated price forthe electronic device based on the condition.

In some embodiments, the kiosk is configured to provide informationabout the electronic device, and wherein the one or more processors areconfigured to invoke a second neural network to determine a final pricefor the electronic device based on the estimated price and theinformation about the electronic device.

Example 10

The system of one or more of examples 1 to 9, wherein the conditioncomprises a physical condition or a cosmetic condition.

Example 11

The system for evaluating a condition of an electronic device,comprising: a capturing device that comprises at least one light sourceand at least one camera, wherein the at least one camera is configuredto capture multiple images of the electronic devices based on one ormore predefined settings, each of the one or more predefined settingsspecifying at least one of: (1) an angle at which the capturing deviceis positioned with respect to the electronic device, (2) a lightintensity of the at least one light source, (3) an exposure setting ofthe at least one camera, or (4) a white balance setting of the at leastone camera. The system also includes one or more processors incommunication with the capturing device, the one or more processorsconfigured to process the multiple images to generate a single inputimage; extract a set of features of the electronic device based on theat least one image of the electronic device; and determine, via a firstneural network, a condition of the electronic device.

In some embodiments, the capturing device is configured to provideinformation about the electronic device, and wherein the one or moreprocessors are further configured to invoke a second neural network todetermine a price for the electronic device based on the condition andthe information about the electronic device.

Example 12

The system of example 11, wherein the condition comprises a physicalcondition or a cosmetic condition.

Example 13

A computer-implemented method for evaluating a condition of anelectronic device, comprising: capturing, by at least one camera of akiosk, at least one image of a first side of the electronic device,wherein the kiosk includes multiple light sources; extracting, by aneural network, a set of features of the electronic device based on theat least one image of the electronic device; and determining a conditionof the electronic device based on the set of features.

Example 14

The method of example 13, comprising: capturing, via the at least onecamera, at least one image of a second side of the electronic devicethat is different from the first side based on at least one lightingcondition generated by the multiple light sources.

Example 15

The method of example 14, comprising, prior to capturing the at leastone image of the second side of the electronic device: flipping theelectronic device such that the light beams are directed towards thesecond side of the electronic device.

Example 16

The method of one or more of examples 13 to 14, comprising: processingmultiple images of multiple sides of the electronic device such that themultiple images have a uniform size; and combining the multiple imagesinto a single image to be provided to the neural network.

Example 17

The method of one or more of examples 13 to 16, comprising: adjustingone of the multiple light sources such that an angle between a lightbeam from the light source and the first side of the electronic deviceis equal to or smaller than 60 degrees.

Example 18

The method of one or more of examples 13 to 17, comprising: determininga model of the electronic device in part based on the at least oneimage; and identifying a cosmetic defect on the electronic device thatis specific to the model.

In some embodiments, the method comprises determining, via a secondneural network, an offer price for the electronic device in part basedon the condition, wherein the offer price is determined further based onat least (1) a predicted resale value of the electronic device, (2) apredicted incoming volume of a model of the electronic device, or (3) apredicted processing cost of the electronic device.

Example 19

The method of one or more of examples 13 to 18, comprising: receiving aninput from a user indicating an acceptance or a rejection of the offerprice; and training the neural network in part based on the at least oneimage and the input from the user.

Example 20

The method of one or more of examples 13 to 19, wherein the conditioncomprises a physical condition or a cosmetic condition.

Some embodiments of the disclosure have other aspects, elements,features, and/or steps in addition to or in place of what is describedabove. These potential additions and replacements are describedthroughout the rest of the specification. Reference in thisspecification to “various embodiments,” “certain embodiments,” or “someembodiments” means that a particular feature, structure, orcharacteristic described in connection with the embodiment is includedin at least one embodiment of the disclosure. These embodiments, evenalternative embodiments (e.g., referenced as “other embodiments”) arenot mutually exclusive of other embodiments. Moreover, various featuresare described which can be exhibited by some embodiments and not byothers. Similarly, various requirements are described which can berequirements for some embodiments but not other embodiments. As usedherein, the phrase “and/or” as in “A and/or B” refers to A alone, Balone, and both A and B.

In other instances, well-known structures, materials, operations, and/orsystems often associated with smartphones and other handheld devices,consumer electronic devices, computer hardware, software, and networksystems, etc., are not shown or described in detail in the followingdisclosure to avoid unnecessarily obscuring the description of thevarious embodiments of the technology. Those of ordinary skill in theart will recognize, however, that the present technology can bepracticed without one or more of the details set forth herein, or withother structures, methods, components, and so forth. The terminologyused below should be interpreted in its broadest reasonable manner, eventhough it is being used in conjunction with a detailed description ofcertain examples of embodiments of the technology. Indeed, certain termscan even be emphasized below; however, any terminology intended to beinterpreted in any restricted manner will be specifically defined assuch in this Detailed Description section.

The accompanying figures depict embodiments of the present technologyand are not intended to be limiting of the scope of the presenttechnology. The sizes of various depicted elements are not necessarilydrawn to scale, and these various elements can be arbitrarily enlargedto improve legibility. Component details can be abstracted in thefigures to exclude details such as the position of components andcertain precise connections between such components when such detailsare unnecessary for a complete understanding of how to make and use theinvention.

In the figures, identical reference numbers can identify identical, orat least generally similar, elements. To facilitate the discussion ofany particular element, the most significant digit or digits of anyreference number can refer to the figure in which that element is firstintroduced.

What is claimed is:
 1. A system for evaluating a condition of anelectronic device, comprising: a kiosk that includes: an inspectionplate configured to hold the electronic device, one or more lightsources arranged above the inspection plate configured to direct one ormore light beams towards the electronic device; one or more camerasconfigured to capture at least one image of a first side of theelectronic device based on at least one lighting condition generated bythe one or more light sources; and one or more processors incommunication with the one or more cameras, the one or more processorsconfigured to: extract a set of features of the electronic device basedon the at least one image of the electronic device; and determine, via afirst neural network, a condition of the electronic device based on theextracted set of features.
 2. The system of claim 1, wherein the one ormore light sources comprises a first subset of light sources and asecond subset of light sources, light beams of the first subset of lightsources and light beams of the second subset of light sources arrangedto be orthogonal to each other.
 3. The system of claim 1, wherein thekiosk further includes: an upper chamber positioned above the inspectionplate, wherein the one or more light sources are arranged within theupper chamber; a lower chamber positioned below the inspection plate;and a second set of light sources positioned within the lower chamberconfigured to direct light beams towards the electronic device throughthe inspection plate.
 4. The system of claim 1, wherein the kioskfurther includes: a flipping mechanism configured to flip the electronicdevice to allow the one or more cameras to capture at least anotherimage of a second side of the electronic device.
 5. The system of claim1, wherein at least one of the one or more light sources are configuredto produce a collimated light beam.
 6. The system of claim 1, wherein anangle between a light beam from one of the one or more light sources andthe first side of the electronic device is equal to or smaller than 60degrees.
 7. The system of claim 1, wherein the one or more cameras areconfigured to capture multiple images corresponding to multiple sides ofthe electronic device under different lighting conditions, and whereinthe one or more processors are configured to process and combine themultiple images into a single input image.
 8. The system of claim 1,wherein the first neural network is configured to output an indicatorindicating the condition of the electronic device.
 9. The system ofclaim 1, wherein the one or more processors are further configured todetermine an estimated price for the electronic device based on thecondition.
 10. The system of claim 1, wherein the condition comprises aphysical condition or a cosmetic condition.
 11. The system forevaluating a condition of an electronic device, comprising: a capturingdevice that comprises at least one light source and at least one camera,wherein the at least one camera is configured to capture multiple imagesof the electronic device based on one or more predefined settings, eachof the one or more predefined settings specifying at least one of: (1)an angle at which the capturing device is positioned with respect to theelectronic device, (2) a light intensity of the at least one lightsource, (3) an exposure setting of the at least one camera, or (4) awhite balance setting of the at least one camera; and one or moreprocessors in communication with the capturing device, the one or moreprocessors configured to: process the multiple images to generate asingle input image; extract a set of features of the electronic devicebased on the at least one image of the electronic device; and determine,via a first neural network, a condition of the electronic device. 12.The system of claim 11, wherein the condition comprises a physicalcondition or a cosmetic condition.
 13. A computer-implemented method forevaluating a condition of an electronic device, comprising: capturing,by at least one camera of a kiosk, at least one image of a first side ofthe electronic device, wherein the kiosk includes multiple lightsources; extracting a set of features of the electronic device based onthe at least one image of the electronic device; and determining, by aneural network, a condition of the electronic device based on the set offeatures.
 14. The method of claim 13, comprising: capturing, via the atleast one camera, at least one image of a second side of the electronicdevice that is different from the first side based on at least onelighting condition generated by the multiple light sources.
 15. Themethod of claim 14, comprising, prior to capturing the at least oneimage of the second side of the electronic device: flipping theelectronic device such that light beams of the multiple light sourcesare directed towards the second side of the electronic device.
 16. Themethod of claim 13, comprising: processing multiple images of multiplesides of the electronic device such that the multiple images have auniform size; and combining the multiple images into a single image tobe provided to the neural network.
 17. The method of claim 13,comprising: adjusting one of the multiple light sources such that anangle between a light beam from the one of the multiple light sourcesand the first side of the electronic device is equal to or smaller than60 degrees.
 18. The method of claim 13, comprising: determining a modelof the electronic device in part based on the at least one image; andidentifying a cosmetic defect on the electronic device that is specificto the model.
 19. The method of claim 13, comprising: receiving an inputfrom a user indicating an acceptance or a rejection of the offer price;and training the neural network in part based on the at least one imageand the input from the user.
 20. The method of claim 13, wherein thecondition comprises a physical condition or a cosmetic condition.